Suchergebnisse

Cognitive pretesting is an essential method of piloting questionnaires and ensuring quality of survey data. Web probing has emerged as an innovative method of cognitive pretesting, especially for cross-cultural and web surveys. The order of presenting questions in cognitive pretesting can differ from the order of presentation in the later survey. Yet empirical evidence is missing whether the order of presenting survey questions influences the answers to open-ended probing questions. The present study examines the effect of question order on web probing in the United States and Germany. Results indicate that probe responses are not strongly impacted by question order. However, both content and consistency of probe responses may differ cross-culturally. Implications for cognitive pretesting are discussed.

Pretesting survey questions via cognitive interviewing is based on the assumptions that the problems identified by the method truly exist in a later survey and that question revisions based on cognitive interviewing findings produce higher-quality data than the original questions. In this study, we empirically tested these assumptions in a web survey experiment (n = 2,200). Respondents received one of two versions of a question on self-reported financial knowledge: either the original draft version, which was pretested in ten cognitive interviews, or a revised version, which was modified based on the results of the cognitive interviews. We examined whether the cognitive interviewing findings predicted problems encountered in the web survey and whether the revised question version was associated with higher content-related and criterion-related validity than the draft version. The results show that cognitive interviewing is effective in identifying real question problems, but not necessarily in fixing survey questions and improving data quality. Overall, our findings point to the importance of using iterative pretesting designs, that is, carrying out multiple rounds of cognitive interviews and also testing the revisions to ensure that they are indeed of higher quality than the draft questions.

Pretesting survey questions via cognitive interviewing is based on the assumptions that the problems identified by the method truly exist in a later survey and that question revisions based on cognitive interviewing findings produce higher-quality data than the original questions. In this study, we empirically tested these assumptions in a web survey experiment (n = 2,200). Respondents received one of two versions of a question on self-reported financial knowledge: either the original draft version, which was pretested in ten cognitive interviews, or a revised version, which was modified based on the results of the cognitive interviews. We examined whether the cognitive interviewing findings predicted problems encountered in the web survey and whether the revised question version was associated with higher content-related and criterion-related validity than the draft version. The results show that cognitive interviewing is effective in identifying real question problems, but not necessarily in fixing survey questions and improving data quality. Overall, our findings point to the importance of using iterative pretesting designs, that is, carrying out multiple rounds of cognitive interviews and also testing the revisions to ensure that they are indeed of higher quality than the draft questions.

A question asking for respondents' sex is one of the standard sociodemographic characteristics collected in a survey. Until now, it typically consisted of a simple question (e.g., "Are you…?") with two answer categories ("male" and "female"). In 2019, Germany implemented the additional sex designation divers for intersex people. In survey methodology, this has led to an ongoing discussion how to include a third category in questionnaires. We investigate respondents' understanding of the third category, and whether introducing it affects data quality. Moreover, we investigate the understanding of the German term Geschlecht for sex and gender. To answer our research questions, we implemented different question wordings asking for sex/gender in a non-probability-based online panel in Germany and combined them with open-ended questions. Findings and implications for surveys are discussed.

Paradata are information about the primary survey data collection process. This guide is intended for survey practitioners who want to collect and use paradata in web surveys. The guideline focuses on a typology and possible applications of web paradata and practical implications regarding the collection, post-processing, and documentation of web paradata.

Das Robert Koch-Institut (RKI) hat die Aufgabe, im Rahmen des Gesundheitsmonitorings Entwicklungen im Krankheitsgeschehen sowie im Gesundheits- und Risikoverhalten in Deutschland zu beobachten. Dazu wird u. a. der Anteil der Bevölkerung erfasst, der die Empfehlungen der Weltgesundheitsorganisation (WHO) zur körperlichen Aktivität erreicht. Ende 2020 hat die WHO neue Empfehlungen für Kinder und Jugendliche herausgegeben, so dass die durch das RKI bisher in telefonischen Fragebögen eingesetzte Frage zur Erfassung der körperlichen Aktivität verändert werden muss, um die aktuellen Empfehlungen abbilden zu können. Dazu wurde sowohl eine Frage für die Eltern 3- bis 15-jähriger Kinder als auch eine Frage für 11- bis 17-jährige Kinder zur Selbstbeantwortung entwickelt. Ziel des Pretests war es, mit Hilfe von kognitiven Interviews zu überprüfen, inwiefern die bereits einmalig eingesetzten telefonischen Fragen und zwei neu entwickelte Fragen zum Einsatz in schriftlichen Fragebögen zur Erfassung der neuen WHO-Empfehlungen in der jeweiligen Zielgruppe geeignet sind. Im Rahmen der kognitiven Interviews sollte die Verständlichkeit der Items sowie spezifischer Begriffe überprüft werden und wie von Eltern und Kindern zur Beantwortung der Fragen Informationen und Ereignisse aus dem Gedächtnis abgerufen werden. Dadurch sollten potenzielle Fehlerquellen oder Schwierigkeiten beim Verstehen der Fragen identifiziert und gegebenenfalls Vorschläge zur Verbesserung entwickelt werden.

Large-scale surveys are increasingly moving from (face-to-face or telephone-based) interviewer-administered to self-administered online modes. To ensure high measurement quality and maximum comparability and equivalence between the source questionnaire and its adaptation - across modes as well as across survey waves - various aspects of question design must be considered and several decisions need to be made concerning question presentation and wording. This survey guideline summarizes good practices on how to transition questionnaires from interviewer- to self-administered web surveys and gives recommendations and examples for major adaptation issues as well as general questionnaire design elements relevant to web surveys. In this context, we focus on the switch from an interviewer-based to an online mode (although mixed-mode designs are also conceivable).

Ausgewählte Fragen der Welle 9 des European Social Survey (ESS) sollten in einem kognitiven Pretest überprüft werden - im Fokus des Tests der Fragen stand die Überprüfung des Unterschieds zwischen den alternativen Formulierungen "fair" und "gerecht" in den jeweiligen Fragen. Ziel des kognitiven Pretests war es, zu verstehen, inwiefern die unterschiedlichen Formulierungen einen Einfluss auf das Verständnis der Fragen und das Antwortverhalten der Befragten haben. Zu diesem Zweck wurde das GESIS-Pretestlabor mit der Durchführung des kognitiven Pretests beauftragt.

Die Studie "FReDA - Das familiendemografische Panel" wird gemeinsam vom Bundesinstitut für Bevölkerungsforschung (BiB), GESIS Leibniz-Institut für Sozialwissenschaften und dem pairfam-Konsortium aufgebaut. Es handelt sich um eine Infrastruktur in Deutschland, die das Gender & Generations Programme (GGP) und pairfam unter einem Dach verbindet. Im Kern von FReDA stehen jährliche Wiederholungsbefragungen von 18- bis 49-jährigen Frauen und Männern sowie deren Partnerinnen und Partnern. Hierdurch soll es Datennutzern ermöglicht werden verschiedene familiendemographische, familiensoziologische und familienpsychologische Fragestellungen zu beantworten. Zur Vorbereitung der FReDA-Welle 3b sollten insgesamt elf neu entwickelte bzw. adaptierte Fragen zum Partnerschaftsstatus, zu Ernährungsstilen, globaler Unsicherheit und zum Vertrauen in Institutionen in einem kognitiven Online-Pretest auf ihre Verständlichkeit hin untersucht werden.